1. Field of the Invention
The present invention relates to a stereoscopic display device and a stereoscopic display method.
2. Related Art
As a stereoscopic video image display device which allows display of a motion picture, a so-called three-dimensional display, ones of various systems have been known. In recent years, particularly, a display device of a flat panel type employing a system where dedicated eyeglasses or the like are not required is demanded highly. As a stereoscopic motion picture display device of such a type, there is known one using a principle of holography, which is difficult to be put in a practical use. A system where a beam controlling element is installed in front of a display panel (a display device) whose pixel positions are fixed, such as a liquid crystal display device, a plasma display device of a direct view type or a projection type is known as a system which can be realized relatively easily.
The beam controlling element is generally called a parallax barrier, and it has a structure that different images can be seen according to an angle change even at one or the same position. Specifically, in case of only a left and right disparity (a horizontal disparity), a slit or a lenticular sheet (or a lenticular plate) is used, and in case that an up and down disparity (a vertical disparity) is included, a pin hole or a lens array is used. The systems using the parallax barrier also are classified to a binocular system, a multiview system, an super multiview system (an super multiview condition of a multiview system) and an integral photography (hereinafter, abbreviated as IP). A basic principle common to these types is the same as that used in a stereoscopic photograph invented about a hundred years ago.
In a simplest binocular system, a certain viewpoint is defined, and a display panel and a parallax barrier are arranged such that different images are respectively seen at the right eye and the left eye of a viewer at the position of the defined viewpoint. A projection plane is provided on the beam controlling element at a distance from the viewpoint to the beam controlling element, and two perspective projection images having projection centers at the right eye and left eye positions are divided vertically for each pixel column and the respective divided pieces are arranged alternately in the display panel. Realization can be achieved relatively easily. In the binocular system, however, there are severe drawbacks that an image is not viewed stereoscopically at positions except for a defined position and a viewing zone is very narrow, that an image appears as a reversed stereoscopic view (pseudoscopy), namely, an abnormal image such that a depth are viewed in a reversed manner when the image is seen at a position moved by a interpupilliary distance (IPD) in left and right directions. There is an advantage that switching between a two-dimensional display and a three-dimensional display can also be performed easily, but application of the binocular system is only an easy application such as a small-sized display.
In order to broaden the narrow viewing zone substantially, a method where the pseudoscopy of the binocular type is avoided by a viewpoint tracking or a head tracking technique has been proposed. As examples of the method, there are a method which switches left and right parallax images from each other, and a method which moves a lenticular sheet in front and rear directions and in left and right directions. Further, a method where an indicator for confirming whether or not an image is out of the viewing zone is provided separately below a screen or around it has been known. In the case of the binocular system, the indicator can allow detection of front and rear and left and right directions. As an example of a viewpoint tracking technique for expanding an effective viewing zone, there has been known an example where an image is changed with a fixed viewpoint according to change in elevation angle of a screen, or an example where a screen angle is tracked (a screen is rotated on a horizontal axis) at a time of viewpoint movement and an image is also changed. There is also an example where the parallax barrier system is not employed but adjustment of a disappearing point of a transparent image, a view line detection/a perspective transformation, and an zooming-in/out are conducted.
In the multiview system, the number of parallax is increased from four to about eight so that the number of positions where an image appears normally is increased. In case that a viewer moves laterally to change his/her viewing angle, he/she sees different images depending on an angle from a stereoscopic display (motion parallax). However, an image which is not continuous but “flipping” appears after a blackout where an angle changes quickly. Further, in the multiview system, the problem about the pseudoscopy still remains.
The super multiview system is constituted such that a parallax image is divided very finely independent of IPD and beams comprising a plurality of parallax images enter in the pupil of a viewer. Thereby, the flipping is cancelled and a more natural image can be obtained. However, since an amount of image information to be processed increases by leaps and bounds as compared with the multiview system, which results in difficulty in realization.
In the multiview system or the super multiview system, there occurs a case of including not only a horizontal disparity but also a vertical disparity. In such a system, however, since the amount of image information to be processed increases by leaps and bounds, it is difficult to realization the system.
The integral photography system (IP system) may be called “integral videography system (IV system)”, “integral imaging system (II system)” or the like, but it is a system which utilizes a lens (a fly's eye lens) similar to a compound eye of an insect as a parallax barrier to arrange elemental images corresponding to respective lenses behind the lenses and perform displaying, where a completely continuous motion parallax can be achieved without including the flipping and beams approximating to a real material in a horizontal direction/in a vertical direction/in an oblique direction can be reproduced. This system is an ideal system which allows a normal stereoscopic view even if a viewer turns his/her face sidelong or obliquely. It is desirable that the elemental image comprises continuous pixels which are not discrete pixels of finite size. However, even if the elemental image is constituted with a collection of discrete pixels such as liquid crystal display panels, a continuous motion parallax with a level which causes no problem practically can be obtained by using pixels with a high fineness of a pixel pitch.
In the IP system which also includes a vertical disparity, namely, a two-dimensional IP system, however, since the amount of image information to be processed increases by leaps and bounds it is difficult to realize such a system. On the other hand, in a one-dimensional IP system which is the IP system where the vertical disparity has been cancelled, since a continuous motion parallax in a horizontal direction can be obtained, a stereoscopic view with a high display quality can be achieved as compared with the binocular system or the multiview system, and the amount of image information to be processed can be reduced as compared with the super multiview system.
The IP system has a viewing zone in front and rear directions boarder than the multiview system, but the one-dimensional IP system has a viewing zone in front and rear direction narrower than the two-dimensional IP system. Since there is no vertical disparity in the one-dimensional IP system, a perspective projection image is displayed on the assumption of a certain viewing distance in a vertical direction. Accordingly, an image is distorted except for a determined viewing distance (in a range including the distance to some extent in front and rear directions) and a correct three-dimensional image can not be obtained. As a result, it is not recognized that there is a large difference between the one-dimensional IP system and the multiview system in a viewing zone in front and rear direction.
The multiview system is the same as the one-dimensional IP system in a point that there is no parallax in a vertical direction. However, the viewing zone in front and rear directions is originally narrow in the multiview system, which does not constrain the viewing zone. In case that the number of parallaxes is as many as about sixteen in the multiview system, the region in the front and rear directions which is out of viewing distance in the multiview system is substantially the same as the one-dimensional IP system, though an image is distorted. That is, this means that a special version of the one-dimensional IP system is the multiview system. In the two-dimensional IP system, since a three-dimensional image of a correct perspective projection is seen in a vertical direction and in a lateral direction according to a viewing distance, no distortion occurs, which results in that the two-dimensional IP system has a viewing zone in the front and rear directions broader than the one-dimensional IP system or the multiview system.
The one-dimensional IP system where the elemental image is constituted with discrete pixels includes the multiview system by definition. That is, of the one-dimensional IP systems, such a special case that the elemental image comprises pixels of a relatively small number of integer pixel columns, a lens accuracy is high (namely, the m-th specific pixel of n parallaxes can securely be seen from any aperture), and a converging (condensing) interval of light beams (crossing lines between a plane connecting the pixel column and a viewing distance plane) is equal to an IPD (62 mm to 65 mm) is the multiview system. Here, a position of a viewpoint (a single-eye) is fixed to a reference position, and a difference in column number between a pixel viewed from an aperture just from the front and a pixel viewed from an aperture adjacent thereto is defined as a pixel column number per elemental image (which may be a fraction instead of an integer) (for example, refer to J. Opt. Soc. Am. A vol. 15, p. 2059 (1998)). A pitch of the elemental image is determined according to an interval between slit centers projected from a viewpoint on to a pixel plane of display panel, but it is not determined from a pixel pitch itself on a display panel.
In the multiview system, pixel centers in a display panel must be positioned on an extension line of both the eyes and all apertures (for example, a slit), so that a high design precision is required. When the eye position is shifted leftward or rightward, it moves at a position where a light shielding portion (a black matrix) between respective pixels can be seen, and when the position is further shifted, an adjacent pixel can be seen (flipping).
On the other hand, in the one-dimensional IP system, a pixel on the display panel can be seen or a black matrix can be seen, or different positions of each pixel can be seen on an extension line of both the eyes and each aperture. An aperture pitch and a pixel width have no relation to each other (ideally, a display which has no pixel such as a photograph is assumed), and a very high design precision is not required. Even if the eye position is deviated, a ratio between pixels where an opening is seen and pixels where a black matrix is seen is not changed so that a flipping does not occur. In this connection, since the aperture pitch when seen from the eye position does not meet integer times the pixel pitch, moire may be seen in case that a black matrix can not be ignored particularly by using a slit.
The one-dimensional IP system which is handled in this specification does not include the multiview system. The definition of the one-dimensional IP system except the multiview system lies in a point that the number of pixel columns in an elemental image is not integer (or a large number which can be assumed to be infinite, and fine), even if there is a position where the pixel columns and a plane connecting apertures forms a crossing line to converge, the converging interval is not equal to an IPD (62 mm to 65 mm) and is different from the viewing distance. In the multiview system, the left and right eyes view adjacent pixel columns, and in the super multiview system, they view pixel columns which are not adjacent to each other. In the IP system, the eyes may view pixel columns which may be adjacent to each other or not. This is because a continuous image where no pixel is within an elemental image is originally supposed in the IP system. Even in either the multiview system or the IP system, when a pixel column group (elemental image) cycle and a pupil (aperture of lens or slit) cycle are compared with each other in a correct design, the latter is shorter than the former without any exception. Incidentally, in extreme conditions unrelated with the practical use, such as a case that the viewing distance is infinite, a case that a screen is infinitely small or the like, the both are identical.
In case that the slit and the display panel are close to each other and the viewing distance is relatively far, the both take values approximating to each other. For example, in case that the viewing distance is 1 m, the slit pitch is 0.7 mm, and a gap which is a distance between the slit and the pixel plane of the display panel is 1 mm, the pixel group cycle becomes 0.7007 mm, which is longer than the slit pitch by 0.1%. Assuming that the number of pixels in the lateral direction is 640, the full width of the slit and the full width of the pixel display portion are deviated from each other by 0.448 mm. Since the deviation is relatively small, even if the pixel group cycle and the pupil cycle are designed to be equal to each other, an image is seen normally at a glance, in case that an image appears only around a central portion (for example, both end portions have a solid color background), or in case that, though a screen size is small, the viewing distance is long. However, the 3-D image can not be seen correctly up to both ends of the screen.
As described above, even in either the multiview system or the IP system, when the elemental image cycle (pitch) and the aperture cycle (pitch) are compared with each other in a correct design, there is a slight difference between the both such as 0.1% or so, but the latter is shorter than the former without any exception. In some literatures or documents lacking in theoretical strictness, there is a description that the both are identical to each other, but these descriptions are erroneous. Further, there are some literatures which are thought to describe that the both pitches are identical to each other in such a meaning as a case that an image has been seen by an eye (a perspective projection centering the position of the eyes), namely, in such a meaning that, since the pupil is positioned on this side of the elemental image, actually different pitch is seen as the same pitch. In general, a difference between the IP system and the LS (lenticular sheet) system is considered to be a difference whether pixels exist on an image plane or they exist on a focal plane. However, in an actual design, and in particular in case that the number of pixels is numerous, a difference between the image plane and the focal plane is 0.1 mm or less, even if there is no aberration. Therefore, it is difficult to discriminate between the image plane and the focal plane regarding the precision and it is also difficult to make discrimination about whether there is presence or absence of convergence of beams in the viewing distance. The IP system in this specification indicates such a constitution that discrimination is not made on the basis of the positions of the image plane and the focal plane and that a viewpoint position in a lateral direction where a normal stereoscopic image can be seen in the viewing distance is arbitrary (continuous). Further, the multiview system in the present specification is not equivalent to the LS system (irrespective of presence/absence of convergence of beams), and it indicates such a constitution that a viewpoint position in a lateral direction where a normal stereoscopic image can be seen in the viewing distance is defined on the basis of an IPD.
Since the viewing distance is generally finite even in either the IP system or the multiview system, a display image should be produced such that a perspective projection image in the viewing distance can be seen actually. It is a usual method to produce a perspective projection image for each crossing point (crossing line) between a line (plane) connecting pixels (a pixel column) and a slit, and a viewing distance plane. In the case of the multiview system, the number of crossing lines between the pixel column and the slit is converged to 16, if the number of viewpoints in the multiview system is 16. Therefore, 16 perspective projection images (all faces) must be produced.
In an ordinary IP system, since convergence of beams does not occur at the viewing distance, perspective projection images (each may be one pixel column instead of all the faces) regarding all pixel column number must be produced. It is thought that, when a computation program is created skillfully, the amount of computation itself is not so difficult from that in the multiview system, but a procedure for creation of the program becomes very complicated. Incidentally, in a special case included in the IP where the slit pitch becomes integer times (for example, 16 times) the pixel pitch (even in this case, the pitch of the elemental image is longer than the slit pitch and it is not integer times the pixel pitch), when a display image is produced by producing 16 orthographic projection images and distributing them for respective pixel columns, a perspective projection image can be seen in a horizontal direction as actually viewed from a viewpoint.
However, an image appearing in this producing method results in a perspective projection in a horizontal direction and an orthographic projection in a vertical direction. Here, a method where the perspective projection is made on a fixed plane along lines converging to one point (a viewpoint, a projection center point) and the orthographic projection is made on a fixed plane along parallel lines which do not converge is employed. However, in the “horizontally-perspective/vertically-orthographic projection”, projection is made on a fixed plane along such lines as converging to one vertical line (which converge in a horizontal direction but does not converge in a vertical direction). In the one-dimensional IP, since a perspective projection image corresponding to the viewing distance can be obtained in the horizontal direction but a vertical disparity is cancelled, a perspective projection image must be displayed on the assumption of a certain viewing distance in the vertical direction.
Accordingly, when the vertical direction and the horizontal direction are combined, there occurs a problem that an image is distorted except for a predetermined viewing distance. In the binocular system or the multiview system, when an image is out of the viewing zone in a front or rear direction, the image becomes a breakup image so that it does not appear to be stereoscopic. On the other hand, in the one-dimensional IP, there is a merit that a front and rear range where an image appears to be stereoscopic is wide, but this merit can not be eventually utilized sufficiently due to distortion occurrence. A precedent has not been found that the projection method or the front and rear viewing distance for an original image in the one-dimensional IP was discussed strictly.
In this connection, in case that an image is photographed as an actual image, which is different from a case that a projection image is produced from a computer graphics, it becomes necessary to simultaneously photograph the image by a camera with a multi-viewpoint and perform such processings as an interpolation, an image conversion or the like.
A method where a convex flexible display plane which is a multiview system is provided, the display plane and a lens are positioned at the same curvature center and the curvature center is set at a position of the head according to detection of the head is disclosed (refer to, for example, JP06-289320A).
As described in detail, since there is not a vertical disparity in the one-dimensional IP system, there occurs a problem that a perspective projection image in a vertical direction must be displayed on the assumption of a certain viewing distance and the image is distorted except for at the distance.